Eecient Stopping Rules for Markov Chains

Let M be the transition matrix, and the initial state distribution , for a discrete-time nite-state irreducible Markov chain. A stopping rule for M is an algorithm which observes the progress of the chain and then stops it at some random time ?; the distribution of the nal state is denoted by ?. We give a useful characterization for stopping rules which are optimal for given target distribution , in the sense that ? = and the expected stopping time E? is minimal. Four classes of optimal stopping rules are described, including a unique \threshold" rule which also minimizes max(?). The minimum value of E?, which we denote by H(;), is easily computable from the hitting times of M. For applications in computing, the most important case is when is concentrated on a single starting state s and is the stationary distribution. We describe a simple, practical stopping rule which achieves target distribution close to in expected time of order Tmix = maxs H(s;). Finally, we give a stopping rule that runs in time polynomial in the maximum hitting time of M and achieves the stationary distribution exactly, even though the transition probabilities of the chain are unknown. Some of the work described herein is joint with David Aldous. 1 Introduction In the past ten years many new applications have been found (see, e.g., 3]) for sampling via Markov chains; these have resulted in polynomial-time randomized approximation schemes for computing the volume of a convex body 12] and counting combinatorial objects such as matchings 13], linear extensions 14], and Eulerian orientations 18]. Typically, in these applications, sampling from an approximately correct distribution is accomplished by walking randomly on a graph for a xed number of steps, after which the distribution of the occupied vertex is nearly stationary. There is no particular reason why such a walk must be run for a xed number of steps; in fact, more general stopping rules which \look where they are going" are capable